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GENERAL THEORY AND METHODS FOR MATRIC FACTORING 


LOUIS GUTTMAN 
CORNELL UNIVERSITY 


Methods are developed for factoring an arbitrary rectangular 
matrix S of rank r into the form FP, where F has r columns and 
P has r rows. For the statistical problem of factor analysis, S may 
be the score matrix of a population of individuals on a battery of 
tests. Then F is a matrix of factor loadings, P is a matrix of factor 
scores, and r is the number of factor variates. (As in current pro- 
cedures, there remains a subsequent problem of rotation of axes 
and interpretation of factors, which is not discussed here.) Meth- 
ods are also developed for factoring an arbitrary Gramian matrix 
G of rank r into the form FF’, where F has r columns and F’ de- 
notes F transposed. For the statistical problem of factor analysis, 
G may be the matrix of intercorrelations, R, of a battery of tests, 
with unity, communalities, or other parameters in the principal 
diagonal. R is proportional to SS’, and it is shown that S can be 
factored by factoring R. This may usually be the most economical 
procedure in practice; it should not be overlooked, however, that S 
can be factored directly. The general methods build up an F (and 
P) in as many stages as desired; as many factors as may be deemed 
computationally practical can be extracted at a time. Perhaps it will 
usually be found convenient to extract not more than three factors 
at a time. Current procedures, like the centroid and principal axes, 
are special cases of a general method presented here for extracting 
one factor at a time. 


Contents 
Introduction. 
Some Properties of Gramian Matrices 
The Generalization of Lagrange’s Theorem 
The Factor Matrices 
The Problem of Metric 
Orthogonal Factors 
Justification for Factoring the Correlation Matrix 
Factoring the Correlation Matrix 
The Special Case of the Centroid Method 
The Special Case of Principal Axes 
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1. Introduction 


This paper develops a variety of new methods for extracting fac- 


tors from matrices. It gives methods for extracting one factor at a 
time and methods for extracting several factors at a time, be they 
oblique or orthogonal. Prevalent methods, like the centroid and prin- 


cipal axes, are shown to be special cases of those developed here. 
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As an example of methods new to factor analysis, it is shown 
that “weighted centroid” methods can be used instead of the centroid 
method, that there is no need to be restricted to +1 weights in sum- 
ming columns in the correlation matrix to determine factor loadings. 
By judicious choice of weights, according to their particular data, 
research workers can reduce mean square residuals more rapidly at 
each stage than by present routines, and without a disproportionate 
increase in labor. 

A variety of new techniques is given for extracting more than 
one factor at a time. It should be feasible to routinize the extraction 
of two or three factors at a time by the formulas developed below. 

Many other possible variations in technique follow from the gen- 
eral formulas established here. 

The methods presented are actually adaptations and generaliza- 
tions of a technique due to Lagrange for reducing bilinear and quad- 
ratic forms (10, p. 68). They are justified by three theorems estab- 
lished below (see §§3, 7, and 8), one of which is a generalization of 
that of Lagrange. These three theorems provide a complete theory 
of matric factoring for factor analysis. 

The formulas are derived rigorously. Experience has shown that 
lack of attention to rigor often leads to unsuspected pitfalls in sta- 
tistical methods for the social and psychological sciences. Factor 
analysis in particular has now reached a stage which makes it neces- 
sary that the discipline be put on a more rigorous foundation. As an 
example, in the Spearman-Thurstone approach there are unsolved 
problems associated with the determination of communalities which— 
when solved—may conceivably vitiate a considerable amount of pub- 
lished empirical work. In this paper, we shall assume that the prob- 
lems of communalities can be taken care of, so that the factoring of 
matrices in the Spearman-Thurstone approach is included. 

Attention to rigor has other practical benefits. To my knowledge, 
no complete justification has been published before for the factoring 
method most widely in use—Thurstone’s centroid method (9). It was 
trying to prove this and related things that led to the uncovering of 
the more general methods presented here. 

Thurstone’s centroid method is partly justified—assuming the 
communality problems are cleared up if communalities are used—as 
a special case of Lagrange’s method. As is shown below, Lagrange’s 
theorem proves that the centroid method does reduce the rank of a 
Gramian matrix by unity at each stage. Completion of the justifica- 
tion is provided by theorems (2) and (3) of §7 and §8 below. 

Because of the practical and theoretical suggestions forthcoming 
from such a treatment, the theory of factoring is presented first from 
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a general matric point of view. The case of the factoring of an arbi- 
trary rectangular matrix is discussed first. This then includes the 
direct factoring of a test score matrix. The methods are seen to ap- 
ply in particular to Gramian matrices, of which correlation matrices 
are a special case. 


2. Some Properties of Gramian Matrices 
In the demonstrations to be given below, certain properties of 
Gramian matrices will be utilized. Proof of these properties can be 
found elsewhere in a systematic treatment of Gramian matrices (3), 
so we shall simply list here those theorems assumed in the present 
paper. The theorems will be stated here only as for real matrices for 
convenience.* The transpose of a matrix M is denoted by M’. 


Definition.+ A Gramian matrix is a symmetric matrix in which 
all principal minors of all orders are nonnegative. (3, p. 49) 

Theorem (a). If M is any rectangular matrix of rank 7, then 
MM’ is Gramian and of rank r. (3, p. 53) 

Theorem (b). For any Gramian matrix G@ of order nXn and 
of rank r > 0 there exists an F of order nX<r (and of rank 7’) such that 
FF’ =G. (3, p. 55) 

Theorem (c). If G is Gramian and nonsingular, then all princi- 
pal minors of G of all orders are positive. (3, p. 56) 

Theorem (d). If M is any matrix whose number of columns 
equals the order of G, where G is Gramian, then MGM’ is Gramian 
(3, p. 75). 

Theorem (e). Every Gramian matrix has a unique Gramian 
square root. (3, p. 121). 

Theorem (f). If MM’ = FF’ = G, where F is as defined in theo- 
rem (b), then there exists an orthogonal matrix U such that 


M=||FO||U (3, p. 68). 


3. The Generalization of Lagrarnge’s Theorem 
In this and the next section, the problem of factoring will be 
dealt with from a purely matric point of view, and special treatment 
will be made later of the case of statistical factors. 
We first need to establish the following lemma. 
Lemma 1. If S is of order n X N and of rank r > 0, then for any in- 


* The treatment in (3) is for general complex Gramian matrices; and actual- 
ly, the theorems for matric factoring of the present paper hold with but a change 
in wording for complex matrices. 

+ An equivalent definition, as can be seen from theorems (a) and (b), is as 
ten a matrix expressible in the form MM’. Other equivalent definitioris are 
possible. 
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teger s such that 0 < s =r there exists an infinite number of pairs of 
matrices X and Y of orders sXn and sXN, respectively, such that 
XSY’ is nonsingular. In fact, all nonsingular matrices of order sXs 
are expressible in the form XSY’. 

The proof follows from a theorem in linear dependence which 
states (1, p. 67) that there always exist nonsingular matrices A, and 
Ay , of orders n X nand N X N, respectively, such that 


I Oj 
A,SA’y = 








: 


I, being the unit matrix of order rX7r . Let X, be the first s rows of an 
A,,, and let Y, the first s rows of a corresponding Ay. Then X,SY’, = 
I,, where I, is the unit matrix of order sXs. Let B and C be arbi- 
trary nonsingular matrices of order sXs , and let 


X= BX,, Y=CY,. 





Then 
ASY’=-8i 490-1 =BC’ . 


Clearly the last member is nonsingular, so that the first member is 
nonsingular. 

Since A, and Ay can be determined in infinitely many ways, so 
can X, and Y,, and all the more so can X and Y. Moreover, since B 
and C are arbitrary, any nonsingular* matrix of order sXs can be 
expressed in the form BC’, hence in the form XSY’. 

The following theorem is basic to the entire discussion. 

Theorem 1. Let S be any matrix of order nXN and of rank 
r>0. Let X and Y be of orders sXn and sXN , respectively (where 
s =r), and such that XSY’ is nonsingular. Then the residual matrix 


S,=S — SY'(XSY’')7XS (1) 
is exactly of rankr —s. 


This theorem generalizes a result due to Lagrange (10, p. 68) 
which can be stated as follows: 
Lagrange’s Theorem: Let S be any matrix of order n X n and of rank 
r>0. Letxandy be row vectors of n elements each and such that 
xSy’ +0. Then the residual matrix 


Sy’ xS 


Ss: = ~ xy 
xSy’ 





is exactly of rankr — 1. 


* More generally, if we do not restrict B and C to being nonsingular, we 
prove that any matrix of order sXs is expressible in the form YSY’. 
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For the proof of theorem (1) (which will prove Lagrange’s theo- 
rem in particular), we first note that lemma (1) assures the existence 
of an infinite number of satisfactory matrices X and Y. Form the 
supermatrix 
é, (XSY')7XS 
M= 











SY’ S 
where /, is again the unit matrix of order sXs, and M is of order 
(n+s) X (N+s). Postmultiplying the first submatric column of M 
by (XSY’')-XS and subtracting from the second column, and using 
(1), yields 





|| 4 0 | 
M,= | I| - 
(sy S| 
Premultiplying the first row of M, by SY’ and subtracting from the 
second row yields 


| 


| 
| |. 


Premultiplying the second submatric row of M by (XSY’)71X 
and subtracting from the first row yields 
| 0 0 || 
| | 

\| 

srs | 
Postmultiplying the second column of M; by Y’ and subtracting from 
the first column yields 


| ds 0 
M.= | 





M,= 





inl | 





lo = 8 || 


Now M, M,, M., M;, and M, differ from each other only by ele- 
mentary transformations, so that they all have the same rank. By in- 
spection, the rank of M, is the rank of S, plus s (the rank of /,) ; and 
the rank of M, is 7, the rank of S. Therefore, the rank of S, is ex- 
actly r—s. 

An important additional fact is that 


XS,=0, S.Y’'=0, (2) 


as is readily verifiable by premultiplying both members of (1) by X, 
and by postmultiplying each member by Y’. Therefore, if it is desired 
to “factor” S, further by the method of the theorem just established, 
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neither X nor Y can be used. The condition that X,S,Y’, be nonsin- 
gular, where the matrices with subscripts now are to replace those 
without subscripts in theorem (1), would not be satisfied if X¥ and/or 
Y is used for X, and/or Y,. 

By choosing X, and Y, of the appropriate rank and such that 
X,S,Y’, is nonsingular, the rank of S, can be reduced again by any 
desired amount, leaving a residual matrix S.. The process can be 
continued until a residual matrix of zero is obtained. 


4. The Factor Matrices 

A matrix S , of order n X N and of rank r > 0, will be said to be 
factored by matrices F and P if F and P are of orders n X r and 
r X N, respectively, and such that S = FP. Then F and P are neces- 
sarily of rank 7, for their ranks cannot exceed r because of their or- 
ders, and the rank of S cannot be greater than either of their ranks 
since § is their product. 

Evidently, if one pair F and P exists, then there is an infinite 
number of pairs of such factor matrices for a given S. If Z is an ar- 
bitrary nonsingular matrix of order r X r, then 


F, = FZ, P,=ZP 


will define a new pair of factor matrices, for F, and P, are of the 
requisite rank and orders, and S = FP, . 

By using theorem (1), we can build up an F and P in stages, inci- 
dentally proving that every S can be factored. In theorem (1), let 


F,=SY'(XSY')?Z,", (3) 
where Z, is an arbitrary nonsingular matrix of order s X s, and let 
P,=Z,XS. (4) 

Then equation (1) can be rewritten as 
S.=S—F,P,. (5) 


Subsequent residual matrices can be similarly treated until, if the 
mth residual matrix is the first to be zero, we have 


S.=S8, — F.P, 
S, = S, — FP; 
0=Sp-1 ee FP» . 


Adding corresponding members of these m equations, we see that 
S=F,P, +F.P.+---+ FiP». 
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Therefore, if we form the supermatrices 


P, 
F=||F,F.--- Fn||, p= || | , 
P, 
then F and P will be factor matrices of S, since they are of order 
nX rand r X N, respectively, by construction, and S = FP. 


5. The Problem of Metric 

In the statistical problem of factor analysis, the matrix S to be 
factored is a matrix of scores of N individuals on n tests. These scores 
are usually expressed in standard form: with zero means and unit 
variances. Or else we can consider the case where unique factor 
scores are subtracted out, so that S represents reduced test scores, 
with zero means again, but with communalities as variances. 

A factor matrix P represents the scores of the N individuals on 
a set of r (common) factors, and these factor scores are usually de- 
sired to be expressed in standard form. The corresponding factor 
matrix F is then said to contain the loadings of the » tests on the 
r factors. 

We can easily show that if the means of the test scores are all 
zero, then the means of the factor scores in P, are all zero. Form the 
row vector of N elements, each of which is unity, 


=||111---1]]. 
To say that the test scores have zero means is to say that 
SS'y=0. mi (6) 
Postmultiplying both members of (4) by 5’y, and using (6), 
P,>'y=0, (7) 


or the s sets of factor scores have zero means. 

Postmultiplying both members of (5) by S’y , and using (6) and 
(7), we see that the right member becomes zero, so that the means of 
the rows in S, are all zero. Hence, the means of the factor scores in 
P, are zero. Similarly, we see that the means of the rows of all the 
remaining residual matrices are zero, and the means of all the re- 
maining factor scores are zero. 

It is quite another matter, however, to discuss the variances of 
the factor scores, and even more so the intercorrelations between fac- 
tors. If the tests are in standard form, then their correlation matrix is 
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1 
R=—SS', 8 
N (8) 


the diagonal elements being unity. If the tests have unique factor 
scores subtracted out, then the diagonal elements in (8) are com- 
munalities. The covariance matrix of the s factor scores in P, (which 
we now know to have zero means) is, according to (4), and using (8), 


1 1 
— P,P’, =— Z,XSS'X'Z', = Z,XRX'Z',. (9) 
N N 


In order for these factor scores to have unit variances, the prin- 
cipal diagonal elements of the covariance matrix must all be unity. 
One way of achieving this—as can be seen from the last member of 
(9)—would be to let Z, be a diagonal matrix whose diagonal elements 
are reciprocals* of the (positive) square roots of the corresponding 
principal diagonal elements of XRX'. Another way would be to modi- 
fy any given Z, by premultiplying it by a diagonal matrix whose ele- 
ments are reciprocals* of the (positive) square roots of the corre- 
sponding elements in the principal diagonal of the last member of 
(9), and using this modified matrix to replace the old Z, . 

In order to obtain orthogonal or uncorrelated factor scores, the 
left member of (9) would have to be the unit matrix. Therefore, in 
order to obtain orthogonal factors, Z, would have to be determined to 
make the last member of (9) the unit matrix. 

A similar discussion holds separately for each of P,, P;,---, Pm. 

There doesn’t seem to be much profit in discussing here methods 
for getting orthogonal factors for the most general case, so this topic 
is not pursued further here. Instead, we go on to the following spe- 
cial method which is of considerable importance. 


6. Orthogonal Factors 


The following lemma will be needed here. 
Lemma 2. If G is Gramian, of order nXn, and of rank r>0, then for 
any integer s such that 0 < s Sr there exists an infinite number of 
matrices X of order sXn such that XGX’' is nonsingular. In fact, 


* The diagonal elements are not zero and are actually positive. To see this, 
note that the rank of P,P’, must be the rank of P, [theorem (a)], so that the co- 
variance matrix is a nonsingular Gramian matrix. Hence all principal minors 
must be positive [theorem (c)]; in particular, the principal diagonal elements 
must be positive. XRX’ is also Gramian since R is Gramian [theorem (d)], and 
is nonsingular since it differs from P,P’, only by nonsingular transformations. 


Hence its diagonal elements are also all positive. 
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all nonsingular Gramian matrices of order sXs‘are-expressible in the 
form XGX’. 

The proof follows from the fact that any symmetric matrix can 
be diagonalized by an orthogonal matrix (10, p. 90). Thus, there ex- 
ists an orthogonal] matrix U such that 
D?, 0 
UGU' = 














0 0 


where D, is a diagonal matrix whose diagonal elements are the (posi- 
tive) square roots of the r positive latent roots of G. [That the non- 
zero latent roots of G must all be positive is well known. To verify 
this directly, note that D?, must be Gramian, since UGU' must be Gra- 
mian by theorem (d).] Form the nXn matrix 


U,D>? A, 
A= 














0 A, 


where U, is an arbitrary orthogonal matrix of order r X 7, A; is an 
arbitrary matrix of order r X (n — r), and A, is an arbitrary matrix 
of order (n — r) X (n— 1). It can be verified by direct multiplica- 
tion that 


| I, 0 || 
A(UGU') A’ = | | ‘ 





0 oO| 


Let X, be the first s rows of AU. Then X,GX’, = I,. Let B be 
an arbitrary nonsingular matrix of order sXs, and let X¥ = BX,. 


Then 
XGX’' = BX.GX'_ B= BI,B' = BB’. 


Since the last member is nonsingular, the first must be. 

Because of the arbitrary submatrices in A , X, can be determined 
in an infinite number of ways (assuming n > 1 if r = n), and hence 
all the more so can X. Since B is arbitrary, any nonsingular* Gra- 
mian matrix of order s X s can be expressed in the form BB’ by theo- 
rem (b), and hence can be expressed in the form X¥GX’ . 

Returning to theorem (1), instead of considering an arbitrary 
Y, let 

Y=XS, (10) 


* More generally, if we do not restrict B to being nonsingular, we prove that 
any Gramian matrix of order sXs is expressible in the form XGX’. 
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where X is-still arbitrary, except again for the restriction that 
XSY' = XSS'X' (11) 


be nonsingular. SS’ is Gramian and of rank r by theorem (a), so 
that the existence of many such matrices X is assured by lemma (2). 
Using (10) and (11) in (3) and (4), and letting* 


Z, = VN(XSS'X’)“, 


we get 
1 
F, =—— SS'X' ( XSS'X’')- (12) 
VN 
P, = VN(XSS'X')“XS8. (13) 
Then 


= PP, = (XSS'X')+XSS'X' (XSS'X') =I. 


The last member shows that the factor scores have unit variances and 
are orthogonal. Therefore (12) and (13) provide formulas for direct 
computations of orthogonal factors. 

Note that, using (8), we can rewrite (12) and (13) as 


F,=RX'(XRX’')* (14) 
P, = (XRX')“XS. (15) 


These are more compact formulas for direct computations of orthog- 
onal factors. 


Some additional facts of some interest, although not very rele- 
vant here, are as follows. Postmultiplying both members of (15) by 


1 
- S’, and again using (8) , we see that 


1 
— P,S'’= (XRX')*XR, 
N 


or, comparing the right member with that of (14), 
2 
— P.S'=F’,. (16) 
N 


Equation (16) states that the factor loadings are the covariances be- 
tween tests and factors. 


* The Gramian square root of XSS’X’ could be used, for example [theorem 


(e)]. 
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An additional bit of information can be got by premultiplying 
(14) by X , which yields 

XF,= (XRX')* = F’,X’. (17) 

Formulas similar to (12) and (13) hold for F,, P.,---, Fm, Pm; 

yielding factors which are orthogonal within each set. However, with 


no further restrictions on the X;, the factors will not in general be 
orthogonal between sets. That is, in general, if 7 ~k, 


P;P’". #90, 
whereas if 7 =k, 


1 Ui 
a al ial 


In the computational methods based on the correlation matrix, 
to be described next, factors will be orthogonal between as well as 
within sets. 


7. The Justification for Computations Based on 
the Correlation Matrix 
Current techniques for factoring operate on the correlation ma- 
trix R rather than on the score matrix §. The usual purpose is to 
find an F of order n X r and of rank r such that R = FF’. 
As we shall see, this implies orthogonal factor scores. In order 
to justify such a procedure, it is necessary to prove the following theo- 
rem: 


Theorem 2. If 
1 
R=— SS’=FF', (18) 
N 
where S is of order n X N and of rank r, and F is of ordern Xr 
(and of rank r), then it is possible to determine a P of orderr X N 


and such that 
S= FP. (19) 


Furthermore, this P is uniquely determined, and it satisfies 
: PP =I (20) 
V ‘ 


The existence of a P satisfying (19) follows from theorem (f), 


setting ail = M, and letting oe P be the first r rows of U. Once 
VN VN 
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the existence of such a P is established,* the fact that it is unique can 
be immediately shown by premultiplying both members of (19) byt 
(F’'F)“F’, yielding 

P= (F'F)“F'S. (21) 


The right member of (21) is clearly unique for a given F. 

That P satisfies (20) follows from the fact that U is orthogonal. 
Actually, we can postmultiply each member of (21) by its transpose, 
divide by N, and use (18): 


1 1 
al i ili ll Ma ial chat il 


This is direct verification that P satisfies (20). 

This last result proves that no matter how the F is built up, the 
corresponding factor scores are automatically orthogonal as long as 
R=FF’. Hence, if such an F is built up by subtests, as is done in the 
following remaining pages, the corresponding factor scores are auto- 
matically orthogonal both within and between subsets. 


8. Factoring the Correlation Matrix 

Current techniques, like Thurstone’s centroid method, usually 
build up a matrix F by computing one column at a time. In the termi- 
nology of factor analysis, one factor is extracted at a time. 

From theorem (1), we can get a method for extracting as many 
factors as we wish at a time, operating directly on the correlation 
matrix. Since this will be a general method for factoring any Gramian 
matrix, let us express it in general terms. 

The fundamental theorem for the method is as follows: 

Theorem 3. Let G be a Gramian matrix of order nXn and of 
rank r >0. Let X be of order sXn and such that XGX’ is nonsingu- 
lar. Then the residual matrix 


G,=G — GX'(XGX’')-XG (22) 


is of rank r — s and is Gramian. 

The proof of the first half of the theorem, that the rank of G, is 
” — s, is given by theorem (1), substituting G@ for S and letting 
Y = X. Lemma (2) assures the existence of infinitely many satis- 
factory matrices X . 


* Holzinger and Harman (4, pp. 289-290) tackle a problem similar to that 
of theorem (2) in attempting to show the relations between sets of factors. How- 
ever, what they actually accomplish is to provide a formula, resembling (21) 
above, for transforming one solution into another if the transformation is pos- 
sible. But the problem is to show that the transformation is possible. Proof of 
this follows from theorem (f). 

} F’F is nonsingular since its rank must be that of F [theorem (a) ]. 














LOUIS GUTTMAN 13 


To prove the second half, that G, is Gramian, let G = F,F’,, 
where F, is of order nXr, according to theorem (b). Then (22) can 


be rewritten as 
G,= FF’, ae FF’ ,X' (XFF'.X') 7 XF F') 
=F,[I — FF’ X' (XF.F'.X')I1XF.)F'o. 


In the last member, denote the symmetric matrix in brackets by H. 
It is readily verified by direct multiplication that H = H’, or we can 
write 

H=HH'. (23) 
From theorem (a), the right member of (23) is Gramian, so that H 
is Gramian. Since G, = F.HF’, , G, must also be Gramian from theo- 
rem (d). 

To obtain s columns of a factor matrix F for G, we can let 


F,= GX'(XGX’')*, 


which is of order nXs, and where the Gramian square root, say, of 
XGX’ is used. Then (22) can be rewritten as 


G,=—G —F,F’,. 


Since G, is Gramian, the factoring process can be reapplied to G,, 
and similarly to subsequent residual matrices, until the remaining 
r — s columns of an F are established. 

As a special case of equation (2), we note that 


XG,=0, (24) 


so that new X; must be used each time for each subset. 

For the case of statistical factors, the Gramian matrix G is the 
correlation matrix R, with usually either unity or communalities in 
in the diagonal. 


9. The Special Case of the Centroid Method 
The calculation of (XGX’)-! may in general constitute the ma- 
jor obstacle to successful application of the preceding method when 
s > 3. It should not be difficult to work out relatively simple proce- 
dures for the case s = 3. 
In the special case where s = 1, the work is extremely simple. 
X becomes simply a row vector, say x, and xGz’ is simply a positive 
scalar number. Hence (xGv’)-? is easily computed by ordinary arith- 
metic. In fact, we can write F, as a column vector, say 


G2’ 
V «Gn j 





A= (25) 
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In the special case where the elements of x are chosen so as to 


be all unity, say 


we have Thurstone’s centroid method, where 
GD’, 
Ae Wyre ° 
V 2nG>'n 
As a special case of (24), we have S,G, = 0, so that 5, cannot be 
used for subsequent factoring, as is well known in Thurstone’s meth- 
od. 
In Thurstone’s method, reflections of axes are made for the ex- 


traction of subsequent factors. This simply means that for factoring 
the residual matrices, vectors of the type 


2, = |[t1 +1 +1... +1]| 


are used, where the signs may be chosen in any manner, just so 
x;Gjx'; #0 (in which case x;G)x'; > 0, since G; is nonnegative def- 
inite). 

The present analysis shows that there is no need to be restricted 
to x; whose elements are +1. Especially if Thurstone’s centroid is 
used for the first factor loadings, it seems that the elements of sub- 
sequent residual matrices will in general tend to diminish more rap- 
idly in the sense of least squares by judicious choice of x; with an eye 
to maximizing «;G;x';/x;x;' (cf. 8). 

For the statistical problem of factor analysis, further investiga- 
tion of this method should yield dividends in practical procedures that 
will reduce mean square residuals rapidly at each stage, without in- 
volving the labor of computing principal axes. An alternative goal 
would be to try to arrive as closely as possible to a simple structure 
—if it exists— in the process of extracting the factors, so as to mini- 
mize a subsequent problem of rotation. 


10. The Special Case of Principal Axes 
As is well known, the method of principal axes minimizes the 
mean square residual at each stage (6, 8). We shall develop this meth- 
od here as a special case of Lagrange’s method. 
Returning to the matrix S and §6, consider the case where s = 1, 
so that X and Y are row vectors, say x and y. Suppose that x and y 
are chosen to satisfy the simultaneous matric equations 


xS= ly (26) 
yS' =x, (27) 
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where 4 and u are scalars to be determined. Substituting 1/1 times 
the left member of (26) for y in (27), and multiplying through by i, 


SS’ = hus . (28) 


This shows that x is a latent vector of SS’ (and hence of R), and du 
is a latent root of SS’. 

Similarly, by substituting 1/u times the left member of (27) for 
x in (26), and multiplying through by u, 


yS'S = iuy. (29) 


Hence* y is a latent vector of S’S corresponding to the same latent 
root ju as does x. 

Equations (26) and (27) can be used as a basis for an iterative 
method for principal axes that operates on the test scores rather than 
on the correlation matrix. This would be of advantage, for instance, 
when the intercorrelations between the tests are not desired, but only 
the major principal axis is sought. In such a case, it may be less work 
to perform iterations directly on the scores rather than to compute 
all the intercorrelations first in order to perform iterations on R. 

Letting 2, y., Ax, and u;, be values of the trial vectors and con- 
stants of proportionality at the kth trial, the iterations take the form 


iY. = LS , MeV = YS’. 


Proof that convergence will take place in general to the major axis 
follows immediately from the proof for iterations on the correlation 
matrix (cf. 6, 8), as inspection of the transition from equations (26) 
and (27) to (28) and (29) will show. 

When s > 1, iterations on the score matrix may not be too fea- 
sible, for the general case, to get several principal axes simultaneously. 

Returning to the factoring of a Gramian matrix—like the corre- 
lation matrix— when s = 1, we see that a principal axis is a special 
case of (25), where now ~ satisfies an equation 


xG = &z, 
&* being a latent root of G@. If x is normalized, 


1Gz’ — Baz’, 


* We have tacitly assumed that \ # ~ 0. (28) and (29) obviously are true if 
A #=—=9. There will be n—r independent solutions to (28) for \ » = 0, and there 
will be N—r for (29). All such solutions, however, are of no interest for the 
present factor problem, since only those r solutions corresponding to the r posi- 
tive latent roots go into the factor matrices. 
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so that (25) can be written in this case as 
fi=é2'. 


Hotelling has discussed iterations on the correlation matrix for 
the case s = 1 (6, 7).* Duncan, Frazer, and Collar (2) have discussed 
this type of iteration for a more general matrix. Horst (5) has gen- 
eralized iterations on the correlation matrix for the case s > 1, and 
proof for this generalization has been provided by Householder and 
Young (8). 
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INTERRELATIONSHIPS OF NUMBER-CORRECT AND LIMEN 
SCORES FOR AN AMOUNT-LIMIT TEST* 


MAURICE LORR 
ADJUTANT GENERAL’S DEPARTMENT 


For an amount-limit test homogeneous as to content and varied 
as to difficulty it is established that an individual’s number-right 
score and his limen score as estimated by the constant process are 
mathematically related. The experimental and the theoretic rela- 
tionship between normal deviate and limen score are shown to be in 
good agreement. It is also found that the two methods of evaluating 
individual test performance yield equally reliable sets of scores for 
the procedures used. Accordingly where the assumptions basic to 
the relationship obtain, the more conveniently computed raw score 
may be considered to be as valid and reliable an index of individual 
test performance as the limen score. The concept of the dispersion 
parameter of the individual as a measure of change or error in test 
score found no experimental verification. Estimates of individual 
variability are unrelated to differences in score on equivalent forms. 


I. Introduction 

The evaluation of individual test performance by the analogue 
of the method of constant stimuli and the related constant process 
was probably first proposed by Thurstone (10). Mosier (7) further 
developed the underlying rationale and pointed out several implica- 
tions for test theory of parameters estimated by means of the con- 
stant process. The precise relationship between the constant method 
evaluation or limen score and the conventional linear evaluation of 
individual test performance, however, remains to be clarified. Like- 
wise the related question of the relative reliability of the two indices 
is yet to be investigated. The objectives of this study are then (1) to 
relate mathematically the conventional number-correct score and the 
corresponding limen score of an individual for an amount-limit test 
and (2) to compare empirically with regard to reliability number- 
correct scores and limen scores. 

In order to illustrate the applicability of the method of constant 
stimuli to mental test data, the usual procedures will be briefly re- 
viewed. For example, in determining a sensory threshold such as the 
two-point tactual limen, an appropriate range of stimuli judged “‘two- 

* The writer gratefully acknowledges Lt. Colonel M. W. Richardson’s invalu- 


able counsel, Dr. H. Gulliksen’s helpful suggestions, and Dr. H. H. Long’s aid in 
administering the tests. 
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point” neither one hundred nor zero per cent of the time is selected. 
Each stimulus is administered a large number of times, in a prear- 
ranged order, to the observer who is instructed to judge the experi- 
ence as “two” or “one.” The responses are then classified and the 
relative frequencies of “two” for each grade of the stimulus scale are 
computed. The stimulus-distance that elicits a response of “two” 50 
per cent of the time is conventionally regarded as the threshold. The 
dispersion of responses in the “two” category constitutes an inverse 
measure of the precision of sensory discrimination. Both parameters 
may, of course, be estimated by the constant process. 

Now consider, instead of a range of sensory stimuli, a represen- 
tative range of test items graded in terms of difficulty for a standard 
group. The items as stimuli are such that the response elicited is 
classed as either “correct” or “incorrect.” Each item may be regarded 
as having a characteristic response-value that differentiates correct 
from incorrect responses and thus each item requires for a correct 
response a given degree of ability as expressed in terms of the stand- 
ard group. This response-value corresponds to the stimulus-value of 
psychophysical discrimination. Theoretically if an individual were 
presented with these items, ordered as to difficulty, and if his re- 
sponses were made without error, correct responses would be made 
up to a certain point on the scale depending upon his ability and in- 
correct responses would be made thereafter. This assumes, in addi- 
tion to the absence of error, that the individual reacts characteristi- 
cally with a given degree of ability to all items of a certain type. 
That is to say, it assumes that there exists for each person a char- 
acteristic response-value called the threshold which is the same for 
all items of a class. Actually, of course, as with sensory thresholds, 
no sharp dichotomy exists between the two possible types of re- 
sponses. In actual practice mixed correct and incorrect responses are 
made to the graded items. Such irregularity of performance is to a 
considerable extent a consequence of the lack of perfect correlation 
between items because of error. Hence the psychological process 
elicited upon presentation of a given item will result in a response 
which is correct or incorrect depending upon whether the ability or 
constant component plus the chance error is greater or less than the 
level of ability required to answer the items correctly. If the com- 
posite process is greater, the response is correct; if it is less, the re- 
sponse is incorrect. 

As the successive sets of items of increasing order of difficulty 
are presented to an individual, a progression of proportions 7, , pz, 
-+ , Pn of correct response is obtained. If the range of item difficulty 
is great enough the proportions will run from 1.00 to 0.00 and, if 
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plotted against the midpoints x, , 2, ---, %, of the corresponding dif- 
ficulty intervals, will tend to give an ogive curve. It is usually found 
that the integral of the normal curve may be taken as representing 
this function of item difficulty. The next step would then be to fit the 
integral of the normal curve to the observed data. The threshold, 
which is not an observed value but must be estimated from all the 
data, may be defined as that difficulty-value of the items where the 
probability is one-half that the individual’s response will be correct 
or incorrect. It represents the characteristic response-value or limen 
score of an individual. The standard deviation of the ogive function 
will be a measure of the individual’s variability or error. Just as the 
precision of sensory discrimination varies from individual to indi- 
vidual with respect to a set of stimuli, so does test variability differ 
from one individual to the next with respect to the sample of items. 
Thus an individual’s goodness of performance may be evaluated and 
described by means of methods and parameters ordinarily applicable 
to psychophysical data. 


II. Theoretical Relationships 

The problem now is to relate mathematically an individual’s num- 
ber correct score and his limen score as estimated by means of the 
constant process for an amount-limit test. The customary procedure 
in evaluating an individual’s goodness of performance on an amount- 
limit test of m multiple-choice items is simply to sum the number of 
items answered correctly. Let X,, X.,--- , Xn be the scores on the 
individual items; they will be either zero or unity. The test score of 
an individual is then given by 


DX =X, + Xe t+-- + X,, (1) 
where >X is the total number of items answered correctly. To cor- 


rect for chance success, which ordinarily is not infinitely small, this 
linear sum may be expressed as 





a 
C ston ae n, (2) 
a-]1 


e~- 2 
where C is the corrected score, SX is the raw score, n is the total 
number of items, and a is the number of alternatives. It is evident 
that if every item were attempted by each individual the proportion 
p- of all items passed and corrected for chance success, would serve 
equally well as an index of test performance. 

This fact is mentioned because the objective is to relate any per- 
son’s number-right score to his limen score on a test in which all items 
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are attempted by every person. It is also evident that p., the total 
proportion of items passed, will be a linear function of C, the cor- 
rected total score. 

Consider now a normal bivariate distribution with linear regres- 
sion, and normal and homoscedastic arrays, as represented in hypo- 
thetical terms in Figure 1. Let each of the variables X and Y be 
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measured in standard units from its mean of zero. Variable X, 
which is quantitatively measured and continuous, represents item dif- 
ficulty or the standard population response to the test items. Variable 
Y , which is dichotomized into two categories, correct and incorrect, 
expresses an individual’s responses in standard units to the same 
items. The bivariate distribution thus represents the hypothetical 
correlation between the responses of a standard group and the re- 
sponses of an individual to a common set of test items. This formu- 
lation, of course, assumes that the trait represented by the dichoto- 
mous distribution is actually a continuous trait, normal in distribution, 
for which we have only categorical information. The normal correla- 
tion surface assumed here for our theoretical development implies 
that each variable is normally distributed (5). In actual practice it 
is likely that such a restriction need not be placed on the shape of 
the continuous variable. 

Let s, measured in standard deviation units from the mean of 
Y , be taken as the point of dichotomy of the variable into two cate- 
gories. It is a measure of the individual’s goodness of performance 
that corresponds to p., the corrected per cent of all items passed, 
and may be secured by recourse to the Kelly-Wood Tables of the inte- 
gral of the normal curve. In Figure 1 the vertical column represents 
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any array in the bivariate distribution. The distribution of scores on 
the Y variable in this array is shown in the normal curve to the right 
of the ellipse. It represents the frequency or third dimension of the 
horizontal slice through the bivariate surface. In this array the theo- 
retical proportion of all responses of an individual to items of a given 
difficulty-value x that are scored as correct lies above s. This is in- 
dicated by the cross-hatched area of the normal curve. This propor- 
tion varies with x , and may be expressed as a function of x and re- 
lated parameters. 

Let the standard score in Y be designated by y. Then the value 
of y corresponding to each value of x is given by the regression equa- 
tion 

dae Vey “, (3) 
where 7,,, the correlation between x and y, is the regression coeffici- 
ent of y on x. For the vertical x array shown, the corresponding pre- 
dicted Y value is y , the mean of the distribution of the array. It is 
shown by the horizontal strip in Figure 1. 

The value of s, the individual’s standard score, expressed in 
terms of the standard deviation of the x array, is 

— 8— Tay & 

Se, (4) 

Oy.z Oy.z 
where o,, is the standard error of estimate, and a is the distance be- 
tween s and y. There will be one array in which a = 0 and the me- 
dian array score will just equal s. This will occur when s = 7,2”. In 
this array, which is represented in Figure 2 by a vertical column 
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through the point of intersection of the line of regression with s , the 
median x array score just equals s. The difficulty value correspond- 
ing to this array is then 
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i=—, (5) 
r 


zy - 
where ¢ designates the x-value of the array in which theoretically 50 
per cent of the responses exceed s and 50 per cent fall below it, i.e., 
the responses are equally often correct and incorrect. 

In agreement with the assumptions, adopted, there will exist for 
each successive array 2; , %2, +++, 4, a theoretical proportion exceed- 
ing the value s. Although the observed proportions will differ from 
the theoretical proportions, it can be assumed consistently with the 
theory thus far applied, that the integral of the normal curve may be 
used as the theoretical curve to be fitted. The value ¢, the threshold, 
is thus seen to be identical with the threshold value estimated by ap- 
plying the constant process to the proportions of correct responses 
secured by the method of constant stimuli. The individual’s number 
correct score on an amount-limit test, if expressed as a normal devi- 
ate, is thus directly related to his limen score as estimated by the 
constant process. 

In order to demonstrate experimentally the relation between an 
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individual’s limen score and his normal deviate score as related by 
(5), the plot shown in Figure 3 and the values recorded in Table 1 
are presented. The statistics given are C, the corrected raw score; 
p-, the obtained proportion of all responses correct; ¢; , the disper- 
sion parameter for an individual; r,, , the correlation between the re- 
sponses of the standard group and the responses of the individual to 
the same items; ¢, the threshold; s, the normal deviate score corre- 
sponding to p.; and rt, the cross-product. 

By procedures more fully described and illustrated in the next 
section, each hypothetical individual was scored as to the number and 
proportion of his correct responses to six subtests of 112 items that 
increase in order of difficulty. The proportions, after correction for 
chance success, were plotted against the midpoints of the median dif- 
ficulty values of the six subtests on arithmetic probability paper. The 
individual discrimination curves were fitted by inspection, and esti- 
mates of the threshold and the dispersion parameter were secured. 
The correlations 7, were estimated by means of the relation 


1 
Voy = . (6) 
ne Si oe 


where ¢; is the obtained dispersion parameter for the individual. 
Richardson (8) has shown that coefficients thus computed are fairly 
close approximations to the more usual biserial coefficients. 

The figure and the table show the close agreement between the 
theoretical and obtained values for the relation between threshold 
score and corresponding proportion correct score expressed as nor- 
mal deviate. The theoretical values are represented by the solid line. 
These results suggest that given an amount-limit test homogeneous 
as to content and varied as to difficulty, the more conveniently com- 
puted raw score would represent the individual’s performance just as 
validly as the more logically justifiable threshold score. The plot also 
serves to explain the correlation of 1.01 between limen scores and 
number-right scores secured by Mosier (7) after correction for at- 
tenuation. It is clear that the two sets of scores bear a proportionate 
relation to each other. 








J 

III. Reliability Data 
1. Methods and Procedures.—The Cooperative Vocabulary Test, 
Form Q, was used for the reliability comparison of the two indices of 
individual test performance. The test is composed of 210 five-choice 
items originally selected for inclusion upon the basis of their differ- 
entiation of the upper and lower 27 per cent of a standardization pop- 
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TABLE I 
Parameters of the Individual Discrimination Function 

















Subject 
No. C ' g "wy t 8 rt 
1 126 56 1.06 .68 28 15 19 
3 68 30 1.02 170 -—-80 -—-.52 —.56 
5 189 84 1.74 50 2.00 99 1.00 
6 178 80 1.48 56 1.50 84 84 
8 17 80 1.00 70 1.30 84 91 
10 22 10 74 .80 1438 -128 -1.16 
11 98 44 1.18 St -2) «415  - 3 
15 38 17 98 71 ~1.00 -.95 -—-.71 
16 37 17 84 16 464-182 -—-95 <~1.00 
18 45 20 2.42 38 -180 -8 -.68 
20 62 28 1.19 64 -95 -.568 - 61 
25 65 29 1.82 48 -106 -55 -—.0 
26 15 33 99 71 -~- 60 -4 -4 
28 80 36 95 72 -50 -26 36 
3 89 40 1.25 62 -2t -235 - 3 
31 94 42 1.03 So «7 +«- - 8 
32 99 44 1.02 so -3 - 2 - 14 
33 92 41 1.04 69 —.35 ~ - 24 
42 109 49 1.13 .66 04 — .08 03 
54 115 51 1.45 57 - .02 03 - £1 
56 135 0 97 71 37 25 26 
57 119 53 1.19 64 .03 07 02 
58 121 54 1.43 57 35 10 20 
61 124 55 1.14 65 31 13 20 
63 135 60 1.21 64 51 25 33 
64 129 58 1.32 60 .50 20 30 
67 70 31 96 72 - 47 -.50 - 35 
69 75 33 1.75 So -# -A ~ 28 
70 77 34 1.71 51 - 71 » él - 36 
72 7 35 1.24 63 — .55 - .39 - 35 
74 85 38 91 “4' «3% - $1 - .19 
15 88 39 1.00 7060 = Sl ~ - 22 
76 89 40 .90 aa | - 25 -— 28 
78 97 43 1.05 68 - 26 ~ — 18 - 18 
80 132 59 99 71 35 23 25 
85 144 64 98 $1 63 36 45 
86 146 65 1.07 .68 65 39 44 
87 147 66 97 71 59 41 42 
92 157 70 88 15 70 52 53 
94 105 47 64 “a = -.08 - 09 
96 102 46 1.16 mo += eM -.10 -.07 
100 164 73 1.09 67 97 61 65 
104 177 79 .96 72 1.21 81 82 
105 167 75 86 16 90 67 68 
y 
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ulation. The test was administered to 500 Washington, D. C., high- 
school seniors. Approximately an hour was required for all persons 
to complete the entire test. 

The 500 answer sheets were machine-scored and the total raw 
scores obtained were corrected for chance success. The answer-sheets 
were then arranged in alphabetical order and a random sample of 100 
was selected tq constitute the experimental group. To transform the 
total raw scores into normalized standard scores and to subdivide the 
transformed scores into a suitable number of class intervals, the pro- 
procedure was as follows. The 400 standard population answer-sheets 
were arranged in order of size of score and split into groups of 50 
each. The total score of the 50 subjects was considered to be a cen- 


troid given by 


— 21 a 22 
“= , (7) 
q 





where x is the centroid or mean deviation of a portion of a unit nor- 
mal distribution included between the ordinates z, and z. which en- 
close the segment at the left and right, respectively ; and q is the pro- 
portion of cases lving in the sector z, — 2 (4). 

The difficulty of an item was ascertained by first securing a 
graphic item count of the number of correct responses to the item of 
the 50 subjects in each of the eight categories. The eight percentages 
for each item (corrected for chance success) were then plotted on 
arithmetic probability paper against the previously computed cen- 
troid values, and a straight line was fitted by inspection. The diffi- 
culty of an item was taken as the standard score of the total test at 
which the item was equally often passed or failed (8). The slopes of 
the resulting item curves were used as a rough index of discrimina- 
tory validity. Items presenting flat curves were eliminated from the 
experimental test. The inclusion of the rejected items in the total 
scores used for computing item difficulty was assumed to have a neg- 
ligible effect principally because of the width of the class intervals 
used in classifying the items. 

On the assumption that difficulty overcome is normally distrib- 
uted, the experimental items were distributed normally into six equal 
class intervals. Each interval, 0.74 sigma units in width, was assigned 
an x-value equal to the midpoint of the class interval. The 112 items 
culled from the original 210 were split into two equivalent experimen- 
tal scales or forms of 56 items each. 

The number and theoretical per cent of the items in each interval 
for forms A and B are given in Table 2. 
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TABLE 2 
- Distribution of Items in the Difficulty Intervals 
Midpoint of Per cent in Number in 
class interval interval each interval 
in standard units 
-1.85 6.8 4 
-1.11 16.7 9 
-0.37 27.0 15 
0.37 27.0 15 
1.11 16.7 9 
1.85 6.8 4 








The midpoints of the tail intervals of the distribution were taken 
as 1.85 and —1.85. Although these are not the correct values, the er- 
ror introduced is probably negligible. 

Since the numbers of easy and difficult items in the difficulty 
scale are few, some means had to be devised to insure the stability of 
response to items in these tail intervals. By combining subjects on 
the basis of total score on the same test form into hypothetical indi- 
viduals, this stability of response can be secured. Accordingly, an 
additional 200 subjects were given the complete form. The answer- 
sheets were then scored on the 112 experimental items. Of this group 
one hundred were paired with the original experimental population 
on the basis of total raw score. Each pair considered as a hypotheti- 
cal individual was scored on both test forms for total number of cor- 
rect responses and the obtained values were corrected for chance suc- 
cess. 

The next procedure was to calculate for each hypothetical person 
the number and proportion of his correct responses to the items in 
each difficulty interval on the two experimental forms A and B. The 
obtained proportions, after correction for chance success (3), were 
plotted on arithmetic probability paper as the ordinate values against 
the midpoints of the corresponding difficulty intervals as abscissae. 
The abscissae were those shown in Table 2; they represent the diffi- 
culty values of the items for the standard group and are expressed in 
standard units. The easier items are negative in sign and the more 
difficult items are positive in sign. The ordinates, similarly, were in 
standard units although the values listed on arithmetic probability 
paper along the vertical axis are percentages. Naturally the greater 
the proportion of items passed by an individual, the more capable the 
individual. It has been assumed here that the normal ogive will rep- 
resent the relation between these proportions and the difficulty values 
of the corresponding items. 
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The individual curves were fitted by inspection and graphic 
smoothing, and estimates of the limen and the standard deviation 
parameters were secured for each hypothetical individual for both ex- 
perimental test forms as well as for the combined form of 112 items. 
The limen was taken to be that difficulty value of the scale where the 
items were equally often passed and failed by the hypothetical indi- 
vidual. The dispersion parameter was estimated by determining the 
two points of the difficulty scale at which 84 and 50 per cent (or 50 
and 16 per cent) of the items were passed. 

To determine the stability of the method of fitting as opposed 
to the constant process, eight curves were fitted by the Miiller-Urban 
procedure as outlined in Guilford (2). The differences were found 
to be slight enough to outweigh the laboriousness of the more prefer- 
able procedure. Application of the constant process to secure esti- 
mates from all the data would have been, no doubt, more admirable 
theoretically. Yet under practically all conditions except theoretic, 
other methods are preferable. It is seldom that the data warrant or 


TABLE 8 
Distribution of Limens and Corrected Scores of Forms A and B 








Limen Scores Corrected Scores 








Class Frequencies Class Frequencies 
Intervals Form Form Intervals Form Form 
A B A B 
-1.99 —1.80 1 0 5 9 1 1 
-1.79 —1.60 0 0 10 14 4 0 
-1.59 —1.40 3 3 15 19 3 4 
-1.39 —1.20 1 3 20 24 2 4 
-1.19 —1.00 a 3 25 29 3 5 
- .99 — .80 6 4 30 84 6 A 
- .79 - .60 9 10 35 39 6 3 
— .59 — .40 cf 6 40 44 9 8 
— .39 — .20 iz 10 45 49 8 6 
- .19 - .00 4 11 50 54 10 14 
01 .20 13 14 55 59 7 10 
21 40 12 14 60 64 11 8 
41 60 8 3 65 69 8 9 
61 .80 8 1 70 74 6 10 
81 1.00 3 3 75 79 4 5 
1.01 1.20 2 6 80 84 4 3 
1.21 1.40 4 1 85 89 4 2 
1.41 1.60 3 1 90 94 3 4 
1.61 1.80 0 2 95 99 1 3 
1.81 2.00 3 3 
2.01 2.20 0 uf 
2.21 2.40 0 1 
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require greater accuracy (1). A clinician, for example, will have 
neither the time nor the inclination to compute an Urban limen. The 
graphic method on the other hand provides greater accuracy than 
that given by linear interpolation. Of course, if the plot is not recti- 
fiable the data are probably not worth application of the more exact- 
ing procedure—or the graphic methods for that matter. 

2. Test Resulis.—The two sets of number-correct scores and the 
two sets of threshold values for the one hundred experimental sub- 
jects were correlated to secure estimates of their respective reliabil- 
ities. The distribution of raw scores corrected for chance success 
and the corresponding thresholds are presented in Table 3 for forms 
A and B. The number-correct scores on forms A and B correlated 
.887, while the limen values correlated .897. The Spearman-Brown 
formula for double length yielded estimates of .94 and .945, respec- 
tively, for the two methods of scoring the experimental tests. The 
comparability of the two experimental forms may be judged by the 
data given in the table below: 


TABLE 4 
Means and Standard Deviations for Forms A and B 








Mean Standard Mean Standard 





Score Deviation Limen Deviation 
Form A 53.3 20.8 0.07 0.79 
Form B 56.4 19.9 0.04 0.83 








These results differ somewhat from those found by Mosier (7) 
for a larger group of heterogeneous items and a smaller population 
(80 subjects). The corrected reliability of the limen scores based on 
odd-even items was .88 compared with a value of .94 for the number- 
correct scores. The findings for this study, on the other hand, indi- 
cate that for the assumptions and the procedures here used the two 
methods of scoring yield equally reliable sets of scores. This conclu- 
sion is consistent with the theory on which the proportionality rela- 
tionship between s and ¢ is based. 

The reliability of a test, of course, represents an average for a 
particular set of individuals. Similarly the standard error indicates 
the average change in any person’s test score as the test is replaced 
by an equivalent form with the same mean, standard deviation, and 
reliability coefficient. Actually there is no reason to suppose that the 
error of a test score is a constant, that is to say, the same for all in- 
dividuals for a given test. It would appear, therefore, that ¢; as ob- 
tained by the method of constant stimuli represents a measure of the 
change or error in an individual test score as successive samples of 
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test items are drawn from the same universe of test items. Mosier 
(6) in particular has developed the concept of the standard deviation 
of errors for a particular individual in a test of items. 

Accordingly an attempt was made to relate e; as a psychological 
characteristic of the individual to (1) the absolute and algebraic 
change in limen score from one test form to the next and (2) to the 
limen score. The value of ¢; for each of one hundred individuals was 
computed graphically on arithmetic probability paper from the ob- 
tained proportions of the correct responses on the combined experi- 
mental forms. The computed values of e; were then plotted against 
the difference t, — t , where ¢, and t, represent the thresholds on the 
two test forms for single individuals. No relation could be observed 
either for the absolute differences or the algebraic differences, as 
plotted against the -e; values. These results would indicate that the 
concept of «; as a measure of change or error in an individual’s score 
is in need of empirical verification or clarification. 

The measures of ¢; were also plotted against the limen scores of 
each subject. Again no relationships were discernible. Here it was 
supposed that there might be variation of the dispersion parameter 
for the individual with the absolute magnitude of score. 

Since the dispersion parameter of a hypothetical individual is of 
doubtful psychological significance, its stability is probably not worth 
investigating. On the other hand, the constancy of «; for an actual 
subject could not be examined satisfactorily because of the relatively 
small numbers of items in the two difficulty scales. When estimates 
of the dispersion parameter were computed for one hundred actual 
subjects on both scales by procedures previously described, the equiv- 
alent form reliability was found to be only .47. 


IV. Summary and Conclusions 

(1) It is established for an amount-limit test of varied difficulty 
that an individual’s number-correct score and his limen score, as es- 
timated by the constant process, are mathematically related. The em- 
piric relation between normal deviate score and limen score is shown 
to be in good agreement with the theoretic relationship. This indi- 
cates that the assumptions basic to the rationale are reasonably well 
justified. The findings also imply that where the requirements of the 
relationship are met the more conveniently computed raw score is as 
valid an index of individual test performance as the limen sccre. 

(2) For the assumptions made and the procedures used it is 
found that the two methods of evaluating individual test performance 
yield equally reliable sets of scores. This conclusion is consistent with 
the established relationship existing between threshold and raw score. 
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Accordingly the conventional linear evaluation of test performance 
may be used with added confidence. 


(3) The concept of the dispersion parameter of the individual as 


a measure of change or error in test score found no verification in the 
data used to test its plausibility. It may be that the estimates of in- 
dividual variability are related to retest changes in score rather than 
to differences in score on equivalent forms. As is well known, test- 
retest reliability differs from equivalent form reliability. 


10. 


aa: 
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THE APPLICATION OF PROBIT ANALYSIS TO THE RESULTS 
OF MENTAL TESTS 


D. J. FINNEY 
ROTHAMSTED EXPERIMENTAL STATION 


The application of the Miiller-Urban constant process to item 
selection, as considered in a recent paper in this journal, is shown to 
be closely analogous to a method now in general use for the analysis 
of insecticidal and other toxicological tests. This method of probit 
analysis gives the maximum likelihood estimates of the unknown 
parameters and, with the aid of published tables, the necessary com- 
putations can be rapidly performed. The present paper contains an 
outline of the method and an illustration of the most convenient 
form of computations for use in analyses of psychometric data. 


1. Introduction 

Methods for the estimation of the limina and precisions of sepa- 
rate items of a mental test have recently been discussed by Ferguson 
(1). In such a test each item admits of only a quantal response from 
each person attempting it: either he passes or he fails. The problem 
considered by Ferguson is thus closely analogous to those arising in 
connection with other material giving quantal responses, and in par- 
ticular with much toxicological experimentation, in which insects or 
other test organisms are exposed to poisons and recorded as having 
died or survived. 

The statistical analysis of toxicological and other dosage-mortal- 
ity data has received much attention in recent years. The chief de- 
velopment has been the method of probits, which, though owing some- 
thing to earlier writers, arose primarily from the papers of Gaddum 
(2) and Bliss (3-6). Tables originally used by Bliss for the numerical 
analysis have been given by Fisher and Yates (7), together with a 
concise account of the arithmetical procedure. The present paper il- 
lustrates the application of the probit method to the data examined 
by Ferguson. 


2. The Probit Method 
When the problem of the toxicologist is compared with the psy- 
chometrical problem discussed by Ferguson, it is seen that the ana- 
logue of the dosage of poison is the ability of a group of subjects, and 
the analogue of the poison itself is the item under investigation. The 
toxicologist measures the effectiveness of the dosage by the propor- 


dl 
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tion of test organisms which succumb to it, the psychometrician meas- 
ures the difficulty of the item by the proportion of subjects who fail. 

If the measure of ability, x, can be so chosen that the propor- 
tion of persons of ability x, in the whole population studied who could 
pass the item is given by the normal integral 


Zo 
— 1 1 
P= - e732 (2-p)?/0? dz, 
|, IV2a 


u may be described as the limen of the item, the level of ability at 
which 50 per cent of subjects will pass. This implies that the propor- 
tion of persons of ability x who are just on the threshold of passing 
the item is normally distributed, s being the standard deviation of 
this normal] distribution. 

The prebit of a proportion P has been defined (5) as the value 
of Y satisfying the equation 


¥-5 
1 
p= { —e'” du; (1) 
7” V 2a 


Y is the unit normal deviate corresponding to P increased by 5, this 
being a conventional addition in order to avoid the occurrence of nega- 
tive values. The probit of P is thus linearly related to x by the equa- 


tion 








1 

Y=5+-(x¢-u). (2) 

The problem of estimating the parameters u, o is equivalent to 

the problem of finding the best fitting linear relationship between 
ability and the probit of the proportion passing the item. This has 
been recognized by Ferguson and is the basis of the “constant pro- 
cess” used by him. Lawley (8) has considered the application of 
Fisher’s method of maximum likelihood (9) to the estimation of the 
parameters, without knowing of the methods of Gaddum, Bliss, and 
others now in general use in the analysis of toxicological data, and 
he has obtained an approximate solution of the problem. A difficulty 
arises when percentages passing in any one group of subjects are very 
low or very high, or when, on account of small numbers of subjects 
or for other reasons, the data are very irregular; the methods of Fer- 
guson and Lawley may then be misleading or inapplicable. Perhaps 
the most elegant form of analysis derives from Fisher’s solution of 
the maximum likelihood equations (Appendix to 5), which has been 
discussed in greater detail by Garwood (10). Fisher and Yates, in 
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the introduction to their Statistical Tables (7), have shown the form 
most conveniently used for the computations. 

The first step is to plot the probits of the observed percentages 
(“empirical” probits) against ability, x; the probits of 0 and 100 per 
cent are infinite and cannot be shown on this diagram. A straight 
line is then fitted by eye to these points, making rough allowance 
for any points at infinity. With experience, and with regular data, 
it is possible so to draw this line that both the limen and the stand- 
ard deviation may be estimated satisfactorily from it, though it is 
generally used only in order to obtain a better approximation to the 
maximum likelihood solution. 

“Provisional probits,” Y , are read from this line for each value 
of x in the data, and from these are derived “working probits,” y, 
where 

Q gq 


ge? + —. 
Z 2 


In this equation Q = 1 — P, where P is related to Y by equa- 
tion (1) and Z is the ordinate to the normal curve, 


1 bi 
6 flee — e-tY-5) : 





q(= 1 — p) is the observed proportion failing the item amongst the 
subjects with this value of «. The working probit remains finite 
when q = 0 or 1 and the empirical probit is consequently infinite. 
For other values of q, if the empirical probit is close to the provi- 
sional line, the working probit has practically the same value. Use of 
the working probit serves to eliminate a bias, arising from the asym- 
metry of the binomial probability law, which is more important the 
greater the divergence of the empirical probit from the provisional 
line. 

An improved estimate of the relationship between x and the pro- 
portion passing is then given by forming the linear regression equa- 
tion of y on x, giving each y the weight nw = nZ?/PQ , where n is 
the number of subjects of ability x. This regression equation takes 
the form, 


Y’=y+b(4—-72), 


x, y being the weighted means of x and y, and } the regression co- 
efficient. If this line differs considerably from the provisional line, 
the process may be repeated using Y’ instead of Y , but usually this 
last equation will give the maximum likelihood estimates of the para- 
meters sufficiently closely as 
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m=x+(5—y)/b 
s=1/b. 


The numbers of subjects in each group who pass the item may 
be compared with the numbers predicted by the values of Y’, a x? 
test being used to test the significance of the discrepancies. If the 
agreement is considered satisfactory, standard errors of the estimates 
of the two parameters and of associated quantities may be derived as 
illustrated in the next section. 


(3) 


3. Application to Ferguson’s Data 

In toxicological applications of the probit method the measure- 
ment of dosage used is either the quantity or concentration of the 
poison or, more commonly, the logarithm of the concentration. When 
investigating the psychometrical problem, Ferguson had a further 
difficulty in that he had no measure of the “ability” of his subjects 
independently of his data. He overcame this ingeniously by using the 
total score for the whole test to obtain such a measure. This score 
includes the results for a particular item, and thus is not indepen- 
dent of the item, but is open to little objection when the test consists 
of a large number of items so that the passing of one has little influ- 
ence on the total score. Ferguson assumed that the distribution of 
“ability” is normal, and he therefore took as x, , the measure of abil- 
ity of a subject A, the normal equivalent deviation of the subject’s 
score, defined as the deviation of the unit normal distribution corre- 
sponding to the percentage of subjects having a lower score than A’s. 
Thus x is subject to errors of estimation, which should not be seri- 
ous in a large sample. 

For convenience, the 216 subjects were grouped into seven 
groups, in intervals of 0.6 of x. Ferguson took the arithmetic mean 
of the end points of each interval as the x-value for the group, but it 
seems preferable to take the mean according to the frequency dis- 
tribution; this may be shown to be 


x= (Z, — Z.)/(P2— P,), 


where P , Z are the area of the normal distribution to the left of the 
end-point of the interval and the ordinate at the point, respectively, 
and may be obtained from tables (7, Tables IX and II). For exam- 
ple, for the interval from 0.3 to 0.9, the area to the left of x = 0.3 is, 
from Table IX, 0.6179, and the ordinate at the point 0.3814; values 
are similarly obtained for «x = 0.9. Hence the mean value of x for the 
interval is 
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(0.3814 — 0.2661) / (0.8160 — 0.6179) = 0.582 . 


The computations for estimating the parameters for Item 1 of 
Ferguson’s paper are given in Table 1. The first four columns show 
x, the number of subjects in each group, 1, the proportion of these 
who pass (as given in Ferguson’s Table 2), », and the probit of p, 
read from Fisher and Yates’ Table IX. These empirical probits are 
plotted against x in Fig. 1 and a provisional line drawn through the 


FIGURE 1 


7.0 
Proportion 
of 
Successes, 
in probits 


6.0 


5.0 








4.0 





-2.0 -1.0 0 1.0 2.0 
Measure of ability, x 
Probit Diagram for Data of Item 1. . 
x empirical probits 
--- Provisional regression line, Y 
—— Fitted regression line, Y’ 


seven points, rough allowance being made for the much greater weight 
to be attached to the points with probits near to 5. The limen may be 
estimated directly from this line as the value of x for which the pro- 
bit is 5; this gives 

m = —1.32. 


Likewise s may be estimated as the reciprocal of the slope of the line. 
In order to proceed with the maximum likelihood estimation, 
values of the provisional probit, Y , are read from the line and entered 
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in Table 1; one decimal place is generally sufficient for Y. Fisher and 
Yates’ Table IX is then used to give the weights, nw, and working 
probits, y. For example, for the highest level of ability Y = 7.5; there 
are 15 subjects in this group and therefore 


nw =15 X 0.0498 = 0.7. 
Also, the working probit is 
y = 7.8543 — (1 — 0.93) X 57.05 = 3.86. 


This is the only one of the seven working probits that is appreciably 
different from the corresponding empirical probit. When the provi- 
sional line is a good fit to all points, there is little necessity to replace 
the empirical by the working probits, but where there is a serious dis- 
crepancy the working probit must be used in order to allow the ob- 


TABLE 1 
Computations for the Calculation of the Probit Regression Equation for Item 1 














Empirical 
x n p probit Y nw y NWwx nwy y’ 
-194 15 0.38 4.56 45 8.7 4.56 -16.878 39.672 4.66 
-1.16 25 0.56 5.15 5.1 15.9 5.15 -18.444 81.885 5.23 
0.58 43 0.70 5.52 5.5 25.0 5.52 -14.500 188.000 5.65 
0.00 50 0.94 6.55 6.0 21.9 6.41 0.000 140.379 6.08 
0.58 48 0.95 6.64 6.5 116 6.68 6.728 76.908 6.50: 
1.16 25 0.96 6.75 69 3.9 6.78 4.524 26.247 6.98 
194 15 0.98 6.48 75 0.7 3.86 1.358 2.702 7.50 
87.7 -37.212 505.793 
Snwax2 Snway Snwy? 
+74.38296 —171.69524 +2961.17353 
—15.78943 +214.61310 — 2917.06453 
S,, = 58.54353 S,,—= 42.91786 S,,== 44.10900 
31.463 
12.646 = x?/5) 
2% — —37.212/87.7 = -0.4248 
y == 505.798/87.7 = 5.7673 
b= 42.9179/58.5485 — 0.7331 


Y’= __— 55.7678 + 0.7331 (a2 + 0.4248) 
= 6.078 + 0.733 x 


a 
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servation to exercise its correct influence on the results; it is there- 
fore preferable to adopt the fully correct method as a routine, rather 
than to consider each case on its own merits. 

The remainder of Table 1 shows the determination of the regres- 
sion equation of y on x. The equation is 


Y’ = 6.078 + 0.733 2 , (4) 


and the values of Y’ are given in the last column of the table. The 
computations could then be repeated using Y’ as the provisional pro- 
bit, but the agreement between this line and the provisional, as shown 
on Fig. 1, is so good as to dispose of any necessity for this*. 

Using S.z, Sz, , Sy, to denote the sums of squares and products of 
deviations of x and y about their means, the expression 


Sy eat Sy] Sas 


may be shown to be equivalent to the y? value attained by comparing 
the observed numbers passing and failing, np and nq, with the ex- 
pected numbers, nP’ and nQ’, where P’ is the proportion correspond- 
ing to the probit value Y’. As is shown in Table 1, this gives 


7° = 12.646, 


with degrees of freedom 5, this being two less than the number of 
groups. Reference to tables of the significance levels of y* (Fisher 
and Yates, Table IV) indicates that this value is significant, imply- 
ing either that the subjects of a group do not give independent re- 
sults, or that a straight line is inadequate to express the relationship 
between probit and ability. 

This method of obtaining y? is rapid and in many cases suffici- 
ently accurate, but it tends to overestimate the value when the ex- 
pected numbers passing or failing in some groups are very small. If 
7° does not exceed the significance level, it is safe to assume that the 
data are in satisfactory agreement with the fitted line, but when 7’ 
is large it is often found that much of it arises from one or two groups 
in which the expected numbers are very small. It is well known that, 
in the latter case, the 7? test is unreliable unless some groups are amal- 
gamated and larger expectations obtained. In the present example, 
closer examination shows the large value of y* to arise chiefly from 
the highest ability group, in which 14 subjects passed as compared 
with 14.9 expected; the expected number failing, 0.1, is thus very 
small. Amalgamation of the three highest groups, in the manner sug- 
gested by Finney (11), removes the unreliability of the test, and 


* If values of Y' are rounded to one place of decimals, the next approximation 
attained is Y” = 6.083 + 0.736%, a trivial change from Y’. 
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gives a 7’? (with 3 degrees of freedom) well within the limits of ran- 
dom variation. 

It then follows that the variances of 7 , the mean probit, and 0, 
the regression coefficient, are 1/Snw and 1/S,,, respectively. From 
the expression for m in equation (3) it may be seen that the variance 
of the limen is 





1 (m — x)? 
V(m) = + 
Saw Bs 
1 (1.047)? 
87.7 «58.54 


= 0.0301 . 


Putting Y’ = 5 in equation (4) gives the new estimate of the limen, 
and its standard error is the square root of the variance; hence 


m = —1.471 + 0.173, 
a value not greatly different from that read directly from the pro- 
visional line. 


TABLE 2 


Comparison of Estimates Obtained by the Constant 
Process and the Probit Method 











Limen Regression coefficient 

Test Constant Probit Constant Probit 
item process method process method 

1 -1.48 -1.47 0.17 0.69 0.73 £0.13 

2 —0.88 0.87 +0.13 1.00 1.05 +0.15 

6 0.03 0.02 £0.10 0.95 1.09 +0.12 
12 0.06 0.04 +0.09 0.76 0.76 +0.11 
20 0.64 0.65 +0.11 0.71 0.73 +0.11 
28 1.79 INT =0.20 0.53 0.56 +.011 








In Table 2 are shown the estimates of limina for the six items 
as obtained by the probit method, compared with the estimates ob- 
tained by Ferguson from the constant process. The differences are 
trivial for these data, but the probit method, in addition to its theo- 
retical advantages, leads also to values of the standard errors of the 
limen. The regression coefficients obtained by the two methods are 
also shown in Table 2, those for the constant process being reciprocals 
of Ferguson’s “standard deviations.” Again the probit method also 
yields standard errors of estimation. The constant process consist- 
ently underestimates this parameter, by comparison with the maxi- 
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mum likelihood values, probably because of zero and 100 per cent 
passes having to be ignored in the constant process.* 

In a short paper it is not possible to consider fully the applica- 
tion of the probit method to this type of data, and extended applica- 
tions to toxicological material may also find their analogues in the 
analysis of psychometric tests. For example, when two or more items 
of a test yield probit regression lines which do not depart significant- 
ly from parallelism, the differences between the items may conveni- 
ently be expressed in terms of the mean ability difference, the hori- 
zontal distance between the lines, in the same way as relative potency 
is used in toxicological work (12). A test for parallelism is obtained 
by comparing the sum of the residual y? values for the lines when a 
separate b is fitted to each item with the corresponding value when 
b is constrained to be the same for all items. Again, if the influence 
of two separate measures of ability on test results is to be investigat- 
ed, the regression method easily extends to the concept of the probit 
plane (13), which may be useful in interpreting the data. 
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* As in all regression problems, any errors of estimation of the independent 
variate (here the ability) will also lead to underestimation of the regression co- 
efficient, irrespective of whether the probit method or constant process is used. 
This effect should not be important when the test consists of a large number of 
items, as the estimate of ability will then have only a small sampling error. 
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ALIGNMENT CHART FOR CALCULATING THE FOURFOLD 
POINT CORRELATION COEFFICIENT 


FREDERIC M. LORD 


The alignment chart may be used to calculate the fourfold point 
correlation coefficient for any fourfold table. If the eategories into 
which each variable of a fourfold table is classified are termed “suc- 
cessful” and “unsuccessful,” then the values needed to enter the chart 
are: A, the per cent of cases successful with respect to the first vari- 
able; C, the per cent of cases successful with respect to the second 
variable; and Q, the per cent of those cases successful with respect 
to the second variable that were also successful with respect to the 
first variable. 

To use the alignment chart: Find the straight line in the central 
part of the chart that corresponds to the desired values of A. Find 
the intersection of this line with the curved line that corresponds to 
the desired value of C. By means of a straightedge connect this inter- 
section with the point on the linear scale at the left that corresponds 
to the desired value of Q. Read the value sought for the fourfold 
point correlation coefficient where the straightedge intersects the FR 
scale on the right. 
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NOMOGRAPH FOR CALCULATING THE FOURFOLD 


POINT CORRELATION COEFFICIENT 
= B'c'-a'p' 
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A SEMI-ANALYTICAL METHOD OF FACTORIAL ROTATION 
TO SIMPLE STRUCTURE 


LEDYARD R. TUCKER 


THE UNIVERSITY OF CHICAGO 


A factorial rotational method is presented which represents a 
compromise between the use of subjective judgment characteristic of 
graphical methods and the routine application of analytical methods. 
At present the analytical methods seem to be inadequate for the dis- 
covery of a simple structure, while graphical methods require more 
subjective judgment. The method herein presented locates the axes 
for subgroups of tests by an analytical method. The judgments used 
in the selection of subgroups are based on graphic data concerning 
interrelation of the factors. 


During the development of multiple factor analysis the rotation 
of the reference axes has risen to great importance. The position of 
the accepted set of axes depends upon the purpose of the individual 
factorial study, as does also the method of rotation employed. In gen- 
eral, that set of axes should be selected which is most useful in solv- 
ing the problem for which the study was devised. Thus the principle 
underlying the method of rotation depends upon the use to which the 
entire study is to be put. For some problems the principal axes are 
the most useful; other studies merely demand that the axes be or- 
thogonal; still others require rotation to positions where the under- 
lying nature of the factors is more easily deduced from the satura- 
tions of the tests. It was for this last case that Thurstone (3) de- 
veloped the principle of simple structure. It is the belief of those 
who employ this principle that axes can be passed through the fac- 
torial configuration so that a large number of tests have vanishingly 
small loadings. 

Although the principle of simple structure seems to be clear-cut, 
the methods of rotation have been cumbersome and difficult to apply. 
However, sufficient experience with these methods has now been ob- 
tained to allow a critical review of their difficulties as well as of their 
merits and thus to permit the development of a compromise between 
the routinely applied analytical methods and the more subjective 
graphical methods. 

The most successful rotational methods have been the graphicai 
ones devised by Thurstone (5, 6). The results from these procedures 
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have, in general, been satisfactory; however, these methods depend 
upon subjective judgments of the investigator and uniform results are 
not obtained by all. Successful solutions of rotational problems by 
these methods depend upon a highly sophisticated sense of geometri- 
cal projection and of fitting lines to points. Further, the computa- 
tional labor seems unnecessarily great, especially when the results 
are carried to a high degree of precision. 

In terms of objectivity of procedure an analytical method of ro- 
tation is to be preferred. Identical results could then be obtained by 
all investigators, and thus the solution could be termed unique. How- 
ever, analytical procedures have a number of inherent difficulties. 
Usually, for each test 7 a value y; is assigned for the corresponding 
loading v; and then the sum of the y’s is maximized, or minimized, 
for each of the rotated axes. Whether a maximum solution or a mini- 
mum solution is desired depends upon the definition of the y’s. An ex- 
ample of a case where a maximum sum of y’s is desired is when 


y,—er"’,” 
and 
> y; =a maximum. 


In this case y; increases when the absolute value of v, decreases and 
has its highest value for a v; of zero. A minimum sum of y’s is de- 
sired when the lowest value of y; corresponds to a zero v;. The im- 
portant point to note is that all tests are entered undifferentially into 
the formulation. Thus the position of the axis is influenced not only 
by those tests which are to have zero or near zero loadings, but also 
by those with high loadings. This results in a shift in the axis from 
the desired position. The amount of this shift varies both with the 
study and with the particular definition of y;. An exceptionally bad 
example is seen in the case where y; is defined as the square of v;. 
When the sum of the y’s is minimized the result is the mean princi- 
pal axis of the entire system where all of the test projections are 
brought to as near zero as possible. Many of the other definitions of 
y; tend toward this same result. 

Thurstone (4) has suggested, as a possible means of overcoming 
the inadequacies of analytical criteria, the consideration of a sub- 
group of tests rather than the entire battery when taking the sum of 
the y’s. This procedure was further developed by Tucker (7), who 
defined the simple structure factor as the mean principal axis of the 
zero subgroup of tests. In general, however, the solutions for any 
assumed subgroup cannot be easily obtained directly and recourse has 
been made to laborious methods of successive approximation. An- 
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other difficulty is in the selection of the subgroup to be treated. If 
the subgroup is taken as those tests which lie within a certain range 
from zero on a trial vector, the resulting solution is merely a tighten- 
ing up of the tests in this range and the trial vector could easily re- 
main a combination of two or three of the simple structure factors. 
This results because all tests high on the two or three factors have 
loadings on the original trial vector outside of the range taken in de- 
fining the subgroup. A more satisfactory method is to solve for all 
factors simultaneously, selecting subgroups from the inter-factor 
graphs. 

Although the use of subgroups has led to improved results, there 
still exists a small tendency for the trial factors to rotate toward the 
mean principal axis of the entire system. This tendency is especially 
pronounced when the variance on the mean principal axis is low and 
this axis can be considered as a residual dimension. The major defect 
of rotation in this direction is that the tests with non-vanishing load- 
ings on the factors have smaller loadings than if this rotation were 
not carried out. There is a loss of variance of projection for these 
tests. 

A solution for this problem of loss in size of the high projections 
is possible with the criterion suggested by Horst (1) who maximized 
the ratio of the sum of squares of the high projections to the sum of 
squares of the low projections; that is, he divided the tests into two 
subgroups, one for those tests with non-vanishing loadings and one 
for those tests whose loadings were to be vanishing. The tests in the 
high subgroup will be designated here by the subscript h, those in the 
zero subgroup by the subscript z. Horst’s solution was then to make 


> Vn? 
h 


=a maximum. (1) 





v2 


& 


»eM 


This criterion seems to have the proper characteristics when the cor- 
rect subgroup has been chosen. However, in applying this criterion 
Horst did not make use of graphical selection of the subgroups, pre- 
ferring, rather, the use of a range of projections on a previous trial. 
The method proposed here makes the solution for all factors simul- 
taneously, using the inter-factor graphs as guides for the selection 
of the subgroups. As a further aid in computation, an approximation 
to Horst’s criterion is used during the trials where the subgroups are 
being selected. 

The fictitious problem used by Thurstone to illustrate rotation 
with extended vectors ( 6) will be used here to demonstrate the prop- 
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erties of the proposed method. Table 1 presents the fictitious simple 
configuration which was rotated to the centroid matrix of Table 2. 
In practice the problem is to start with a centroid matrix such as is 
given in Table 2 and rotate to the simple configuration. In this case, 
however, it is convenient to have the exact solution given in’Table 
1 in order to judge the goodness of the rotations. 

The first problem in applying the method is that of obtaining an 
initial set of trial subgroups. These subgroups may be obtained by a 
cluster analysis of the original correlations, by inspection of the cen- 
troid factorial matrix, or, as is suggested here, by a cluster analysis 
of the first factor residuals. It is assumed that the configuration has 
not only a simple structure, but also a positive manifold. When this 
is true, as with most test data, the first centroid axis passes some- 
where in the middle of the positive manifold and the test vectors are 
projected down into the space orthogonal to this axis. Not all the 
residual correlations are positive. In fact, positive residuals occur 
only for those pairs of tests that extend in the same general direc- 
tions and thus have loadings on the same factors. 

Table 3 gives the first factor residuals for the illustrative prob- 
lem. It is noted that the highest residual in the table is between vari- 
ables 2 and 14. These two tests are taken as the starting point for the 
first cluster. The next test added to this cluster is test 4, which corre- 
lates .45 and .30 with tests 2 and 14, respectively. Succeeding tests 
are added to the subgroup in such a way that all of the residual inter- 
correlations are positive. The tests in this cluster are listed in col- 
umn A of Table 4. 

Cluster A accounts for a number of the positive residuals. The 
next cluster is started by finding the highest residual that has not 
been accounted for by cluster A. This residual is between variables 
1 and 13. Cluster B, of Table 4, is found starting with these tests. 
All of the residual intercorrelations within this cluster are also posi- 
tive. The other clusters are found in a similar manner. Since the 
problem has five dimensions as shown by the five columns of the cen- 
troid matrix of Table 2, five clusters are taken. In general, as many 
clusters should be taken as there are dimensions in the factorial ma- 
trix. These clusters will be used as the first assumed subgroups of 
“high” tests. 

There is one other preparatory set of computations to be made 
before starting on Trial 1. The centroid factorial matrix of Table 2 
must be premultiplied by its transpose, thus producing the matrix 
P;; that is, 


Pat oF, (2) 
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This product matrix for the illustrative problem is given in Table 5. 
The diagonal entries are the sums of the squares of the correspond- 
ing columns of F, and the off-diagonal entries are the sums of cross- 
products between columns of F,. The inverse of P is also found and 
is listed in Table 6. P-' will be used in the calculations for each trial. 

The computational work for Trial 1 is shown in Table 7. The 
matrix W has a row for each of the tests and a column for each fac- 
tor. The factor A in matrix W corresponds to cluster A of Table 4. 
Each of the tests in this cluster has an entry of unity for this factor. 
All other cells in the column are left blank to indicate entries of zero. 
The other columns of W correspond to the other clusters. 

The first step in Trial 1 is to find the matrix B by the equation: 


B=F'.W. (3) 
The matrix LZ is then found by: 
L=P"“B. (4) 


The columns of Z are then normalized to the corresponding columns 
of A. The matrix C is found by: 


C=A' A. (5) 


This matrix shows the cosines of the angles between the trial factors 
and should be closely inspected to see if any pair of the factors are 
nearly identical. If this should occur, a new cluster should be tried. 
This is shown by a high positive side entry. The diagonal entries 
should be unity if the normalizing has been done properly. The last 
computational step of the trial is the finding of the matrix of test 
projections V by the equation: 


VFA. (6) 


When the matrix V has been obtained, a plot is made for each pair 
of columns. In the present example this results in ten graphs as 
shown in Figures 1 and 2. The foregoing procedure is followed for all 
trials except that the entries in the W matrix are derived from the 
diagrams rather than from the table of first factor residuals. 

The entries in the W matrix of Trial 2 shown in Table 8 were 
derived from the graphs for Trial 1 shown in Figures 1 and 2. The 
graphs of factor A with the other factors were inspected in setting 
up the column A of W. The points 2, 14, 4, 3, and 23 are plainly high 
on factor A in all four graphs involving A. On the graph A-D, 
points 6 and 8 clearly seem to deserve high loadings on A. The con- 
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figuration of points in the E direction on the A-E graph seems too 
wide for all of the tests to have vanishing loadings on factor A and thus 
tests 12 and 24 are included in the high group on factor A. The case 
of tests 1 and 5 in the B direction on the A-B graph is more ambigu- 
ous; however, it was decided to include them also in the high group 
on factor A. Tests 21 and 25 on the A-C graph were left in the zero 
group on factor A. The tests placed in the high group have unities 
entered in the A column of the W matrix of Table 8. The cells for the 
tests in the zero group are left blank. Each of the other columns in 
W were obtained in the same way by inspecting the graphs for each 
of these factors. The selection of the subgroups is the high point in 
the procedure for Trial 2, since the computational procedure is iden- 
tical in form to that outlined in the preceding paragraph for Trial 1. 

When the computations for Trial 2 are completed and the graphs 
made, it is found that all of the subgroups are satisfactory. Two of 
these graphs are illustrated in Figure 3; the graph A-E seems to be 
the poorest of the ten and C-D the best. In order to tighten the points 
to the planes a third trial is made in which the entries in the W ma- 
trix of Table 9 for the high tests are taken as equal to the v’s of Trial 
2 rounded off to one decimal. Again the cells for the zero group are 
left blank. The computations are again the same as for Trial 1. The 
results of this trial are very near the ideal solution indicated in the 
original simple configuration of Table 1 except that the columns are 
in a different order. The order of the columns is of no consequence 
since they can be renamed and their order changed to any order de- 
sired. The graphs of Figure 4 illustrate the tightness of fit that has 
been obtained. All that is necessary to tighten the fit of the planes 
still further is to continue to another trial with improved weights in 
the W matrix taken from the present set of loadings in the V matrix 
of Table 9. However, since this was merely a repetition of the pro- 
cedure already presented, it was not carried out. 

A profitable variation in the procedure after Trial 1 is presented 
in Table 10 where the work of Trial 2’ is given. The difference be- 
tween Trial 2 of Table 8 and Trial 2’ is in the use of differential 
weights in the W matrix of Trial 2’. These weights are assigned to the 
high tests from inspection of the graphs for the first trial which are 
presented in Figures 1 and 2. This was done at the time that the 
tests were placed in the high subgroup. Since the weights depend 
upon subjective estimates of final size of loadings from these graphs, 
it is probably best not to use too complicated or extensive a set. 
Therefore, it was decided to restrict these weights to the digits 1, 2, 
or 3. Tests 2, 14, and 4 are plainly higher than all of the other tests 
on factor A and are, therefore, given weights of 3. Tests 3 and 23 
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are given weights of 2, as are tests 6 and 8. These latter two tests 
seem from the graph A-D to have disproportionately low loadings on 
Trial 1. The other four tests in the high group, tests 1, 5, 12, and 24, 
are given weights of unity. The results of this trial are closer to the 
exact solution than are those of Trial 2, as will readily be seen by 
inspection of the V matrices of the two trials. 

In order to gauge further the amount of discrepancy of each 
trial from the exact solution, the differences between the loadings in 
each trial and those in the exact solution were found and frequency 
distributions of the absolute values of these errors made. These 
distributions are shown in Table 11. It will be noted that each of 
the trials in the series 1, 2, and 3 are improvements over the pre- 
ceding trials. Trial 2’, where subjective differential weights are used, 
is definitely better than Trial 2, being almost as good as Trial 3. The 
use of simple differential weights is recommended because of this 
gain. Further reassurance can be obtained from the fact that errors 
in these weights are not a too serious matter, for the goodness of re- 
sults should not regress any further than if all of the tests were 
given unit weights as in Trial 2. It merely retards progress. 

The advantage of using the graphs in selecting the subgroups 
can be illustrated by the initial selection of a high subgroup which 
includes tests high on two factors, B and C in this case. In Table 12 
an alternate factor B’ for Trial 1 is presented. It is to be imagined 
that this factor replaces the B factor in Table 7. The graphs between 
this new factor and the other four factors of Trial 1 are presented 
in Figure 5. The most profitable diagram for selecting the next high 
subgroup is B’-C , where there are no tests holding the B plane in the 
direction of C. A line for the new plane can be drawn on this dia- 
gram and tests selected which are farther from it. In this way the 
factors are guided to their proper solutions. 

The computational procedure can be derived by either of two 
mathematical rationales. In the first of these the criterion is defined 
as: 

Yin = (Win — Kk Vin), (7) 
and 

> yj? =a minimum. (8) 

J 
The subscript p is used to designate rotated factor; thus wj, is the 
entry for test 7 and factor p in the W matrix. v;, is the resulting pro- 
jection of test 7 on factor p. The w,,’s are in this rationale to be 
thought of as proportional to a hypothetical set of loadings which 
might be obtained at the solution. The y;,’s are the errors in fitting 
an actual set of loadings to these hypothetical ones. k is used so as to 
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consider only the relative order of magnitude of the v;,’s._ This solu- 
tion has been derived by Mosier (2), where the w;,’s were obtained 
from previous factorial studies. In this case they are derived from 
inspection of the configuration of the battery being investigated. 
Equation (8) is a least-squares statement for the errors yj, . 
A matrix equation for the v;,’s is given in (6). In summation 
form this is: 
Vip po Dim Amp (9) 


where @;» is the entry for test 7 and reference factor m in the origi- 
nal factor matrix F,. Am, is the direction cosine for rotated factor 
p from the reference factor‘m. In order to facilitate the derivation 
it is convenient to define: 


Ujp = k Vip; (10) 
for, then 

Ujp =k > Ajw. Amps 

Ujp = = jm k Amp» 
or 

Ujp => Qjm bay» (11) 
where 

Imp = Ke Amp « (12) 


The /,,,’s are direction numbers and are mutually independent. Equa- 
tion (7) then becomes 

Y jp = Wip — Ujp. (13) 
In order to find the minimum solution of equation (8), the partial 
derivatives with respect to the l,,,’s are found and equated to zero. 
That is: 











0> Yip 
j 0 Yin 
“Oh la 
using equation (13), 
0D Yip 
= Wip — Uj = ’ 
2 Ip p ip 2 Imp 
and by (11), 
0 = Y’ ip 
7 
= 22> (wip — Dd Qu lap) (— Qjm) = 0. (14) 
o bap j M 
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M is used as an alternative subscript for m. Equation (14) can be 
simplified to: 


— > Wip Gm + E> Aju lap dim =, 


j 7 = 
or 
> Win Um = > Ua Lup Vim, 
j iM 
or 
> Win Vim => lap > Vu Um. (15) 
j u i 


Equation (15) in matrix form is: 
F, W=F, F.L, 
or by equations (2) and (3), becomes: 


B=PL, 
or 
P“B=L. (4) 


Equation (4) is the solution for minimizing the discrepancies be- 
tween a hypothetical set of values given in W and values proportional 
by columns to the entries in V. The factors approach the desired 
rotated solution as the w;,’s are brought closer to the final v;,'s. The 
discrepancies of the tests in the zero subgroup are constantly mini- 
mized since their w’s are always set at zero. 


The second rationale from which the computational procedure 
can be obtained is slightly more complicated than the first, but throws 
some additional light on the nature of the solution. This rationale 
starts from Horst’s criterion in equation (1). If all of the tests are 
either h tests or z tests, 


Dd Vip => V7 + E V7293 (16) 
,] h z 
then 
> Vp => Vin — DS V7np, 
z j h 
and 


> V7np > V* np 
h 


h 





Se EPn—- Te’ 
z j h 
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or 
D V7 rp 
: : (17 
a ) 
> V7 sp >> V? ip 
2 ] 
> Vnp 
h 
It is evident that when 
=a maximum 
> Vp ‘ 
> v5 
j 
—— — 1=aminimum; 
> Vp 
h 
then 
30, =a maximum. (18) 
ip 
j 


This last equation provides a simpler basis for the development of 
both the exact solution and the approximate ones. 

As in the first rationale, it is easier here to deal with direction 
numbers, ln)’s, and values, u;,’s, proportional to the v;,’s. Substitut- 
ing equation (10) in equation (18) and cancelling the (1/k*)’s in the 
numerator and denominator, 


> wap 
h 


=a maximum. (19) 





> U? jp 
j 


It is convenient now to define a condition placed upon the l,,’s; that 
is, it is defined that: 
> wip = 1. (20) 
j 


With this condition it is necessary only to make 
> u?,p = a maximum. (21) 
h 
However, instead of continuing to the exact solution, consider the fol- 
lowing equation: 
SD Ujp Wjp = a Maximum. (22) 


I 
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When the wj,’s are zero for the z tests and equal to the u,’s for the 
h tests, this equation becomes the same as (21). However, when the 
Wr, S are plus or minus one, the signs being taken as the same as the 
Un’ S, equation (22) becomes 


> | % | =a maximum. (23) 
h 


Thus, with proper selection of the w,,’s, the weighted sum of the 
Ujp's becomes either the sum of the squares of the w,,’s or the sum of 
the absolute values of the w,’s. This latter squared is used as ap- 
proximately proportional to the former. That is: 


[S |p|]? = ¢ > wry. (24) 
h h 


The solution of equation (22) under the condition of equation 
(20) is accomplished with the use of LaGrange multipliers y,. Thus 








0D Ujp Wp 0(S u?jp — 1) 
j j 
~ =. (25) 
= 
Then 

é Ujn 0 Ujp 

D> Wip —— + 2 yp Dd Ujp ——— 0; 
] mp j mp 


or, using equation (11) 

D Wip Vim + 2 yd ZX Aju lap im = 0, 

i j M 
or 

> Wjp Lim = — 2 Yom Lup = Aju jm. (26) 

j M ] 
However, since each column of L is to be separately normalized, the 
multipliers (—2 y,) can be dropped and equation (26) becomes iden- 
tical with equation (15) and reduces to equation (4) in a similar 
manner. 

Since equation (24) is usually a good type of approximation, it 
was expected that the use of w,,’s equal to unity should give good 
results. This expectation is at least partially born out by the results 
illustrated in Trials 1 and 2 presented in Tables 7 and 8. Better ap- 
proximations can be obtained by using differential weights as in Trial 
2’ in Table 10 since these w,,’s approach closer to being proportional 
to the solution ¥;,’s. 
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Although the method was illustrated with a problem having a 
positive manifold, it is not restricted to suck a problem. Tests with 
negative loadings can be entered into the high group by using nega- 
tive w,,’s. The only portion of the procedure that is not generally ap- 
plicable is in the selection of the first trial subgroups. Some other 
principle than the one used here may have to be used in the case where 
there are tests with significant negative factor loadings. 

The author is indebted to Professor L. L. Thurstone for his many 
suggestions and criticisms during the development of this rotational 
method. The present study is issued as a report from the Psycho- 
metric Laboratory at the University of Chicago and has been made 
possible by research grants from the Social Science Research Commit- 
tee and from the Carnegie Corporation of New York for the Psycho- 
metric Laboratory. A further acknowledgment is due to Mrs. Mary 
S. Carroll for her editorial assistance during the preparation of the 


manuscript. 
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TABLE 2 
Centroid Factor Matrix, F, 


59 











I II Hil IV Vv 

1 o1 —.19 53 33 —.22 
2 37 49 04 —.08 38 
3 65 47 —.03 —.30 —.08 
4 51 31 37 —.48 15 
5 60 —.27 OT 32 02 
6 61 18 —.21 me 45 
4 36 —.35 —.30 .10 .54 
8 02 —.03 04 05 69 
9 .54 07 —.56 .26 22 
10 69 —.06 —.26 45 —.13 
11 67 —.08 —.46 —.31 05 
12 57 —.07 00 —.57 13 
18 44 —.39 39 39 —.39 
14 51 .66 25 06 21 
15 64 —.44 —.11 —.30 04 
16 -54 —.35 —.28 —.42 24 
17 56 —.52 —.04 31 .26 
18 52 —.37 27 —.03 —.42 
19 i) —.12 —.07 —.67 —.23 
20 56 07 —.07 45 —.45 
21 36 44 —.44 25 —.25 
22 65 —.13 .06 18 —.51 
23 67 3) 24 33 —.16 
24 .58 —.07 25 —.48 —.20 
25 49 40 —.AT .00 —.33 
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TABLE 4 


Positive Clusters of First Factor Residuals 















































A B C D E 
2 1 21 7 12 
14 13 25 8 19 
4 18 9 17 24 
3 5 20 6 4 
23 22 10 9 16 
20 16 
23 
TABLE 5 
P=F,F, 
I II Ill IV V 
I 7.5128 —.0857 —.0568 0768 —.0940 
II —.0857 2.6564 —.0002 —.0707 1038 . 
III —.0568 —.0002 2.5033 .0704 —.2887 
IV .0768 —.0707 0704 2.8753 —.3891 
Vo —..0940 1038 —.2887 —.3891 2.5289 
TABLE 6 
p-1 
I II a _ Vv 
I 1333 0040 0087 —.0029 .0048 
II 0040 3773 —.0018 0078 —.0144 
Ill 0037 —.0018 4049 —.0039 0458 
IV —.0029 0073 —.0089 3554 0538 


Vv 0048 —.0144 .0458 0538 4097 
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TABLE 11 


Distributions of Absolute Discrepancies 











Discrepancy Trial 1 Trail 2 Trail 2' Trial 3 

26 1 

25 1 

2A i 

.23 
22 2 
21 

.20 5 

19 1 

18 2 

7 

16 2 

15 8 
14 4 
13 3 
12 4 4 

aa 2 

10 4 8 

.09 3 5 1 
.08 6 8 4 
07 10 9 1 

06 15 7 1 ie 
05 7 16 4 2 

04 10 19 8 5 

03 11 11 21 20 

02 10 10 22 16 

01 12 16 33 50 

00 1 12 84 32 
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TABLE 12 
Plane B’ for Trial 1 

W B ye 
1 I 2.57 1 49 
2 II al 2 —.07 
3 III —.03 3 25 
4 iV t1Al 4 —.13 
5 V —1.25 5 238 
6 6 LT 
7 L 7 —.12 
8 8 —.15 
9 I 33 9 35 
10 1 II BS i 10 .69 
11 III —.07 usd 13 
12 IV 48 12 —.17 
18 Vo —.43 18 58 
14 14 24 
15 A 15 03 
16 16 —17 
17 I 47 17 21 
18 II say 18 39 
19 III —.10 19 —.11 
20 1 IV 61 20 83 
A | V —.61 21 59 
22 1 22 70 
23 1 C 23 65 
24 24 07 
25 A .05 25 55 

B00 

C -74 

D —.20 

E —.39 








*The entries in the V matrix were computed on the 
Matrix Multiplier (8) and may be in error by .01. 
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RESEARCH NOTE 
IN SEARCH OF NEW TESTS 


We should like to ask the assistance of psychologists in our search 
for a particular kind of test. Most tests are of the kind that give a 
unimodal distribution in which even the poorest subject makes some 
appreciable performance and in which the ratio between the best and 
the poorest is about 2 to 1, or 3 to 1. Other things being equal it is 
expected that if a test is factorially relatively pure (low complexity) 
then the individual differences will be conspicuous. Some subjects 
endowed with the factor involved will find the task easy and they will 
even take some pleasure in doing it. Others who are not so endowed 
will find the task not only difficult and disagreeable but they may even 
refuse to continue trying. We are looking for tests which can be 
easily explained to untutored adults and which show these gross indi- 
vidual differences. It is not a question of whether the ability that is in- 
volved is educationally or socially important. That is irrelevant. Nor 
do we care at all whether the distribution is normal or skewed. That 
is entirely irrelevant. We are looking for tasks that are easily ex- 
plained to adults and which show gross individual differences. Such 
tests are likely to be relatively pure factorially and they should be 
significant in isolating the fundamental parameters of mental func- 
tion. Only occasionally do we find such tasks. 

We shall appreciate it if others who discover such simple tasks 
will let us know so that more tests of that kind may be incorporated 
in factorial studies. 

L. L. THURSTONE 
University of Chicago 
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